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Abstract 

Traditionally, the performance of ocr algorithms and systems is based on the recognition of isolated 
characters. When a system classifies an individual character, its output is typically a character label or a 
reject marker that corresponds to an unrecognized character. By comparing output labels with the correct 
labels, the number of correct recognition, substitution errors misrecognized characters, and rejects 
unrecognized characters are determined. Nowadays, although recognition of printed isolated characters is 
performed with high accuracy, recognition of handwritten characters still remains an open problem in the 
research arena. The ability to identify machine printed characters in an automated or a semi automated 
manner has obvious applications in numerous fields. Since creating an algorithm with a one hundred percent 
correct recognition rate is quite probably impossible in our world of noise and different font styles, it is 
important to design character recognition algorithms with these failures in mind so that when mistakes are 
inevitably made, they will at least be understandable and predictable to the person working with the 
program. 

Index Terms - Hidden Markov Model, Optical Character Recognition, Offline Handwritten 



1. INTRODUCTION 

Since the advent of digital computers, there has 
been a constant effort to expand the domain of 
computer apphcations. Some of the motivations for this 
effort come from important practical needs to find more 
efficient ways of doing things and some from the sheer 
challenge of building or programming a machine to do 
things that machines have never done before. Both of 
these motivations can be classified in the area of 
artificial intelligence, called machine perception. 
Character recognition is an important topic in pattern 
recognition, which is the backbone of machine 
perception. Invention of scanners and improvement in 
scanner technology has enriched the applications of 
automated character recognition. 

Handwritten data can be generated in two 
different ways, on-line and off-line. In the former, the 
data were captured during the writing process by a 
special device Stylus pen on a tablet. In the latter, the 
data are captured by scanner after the writing process is 
over. In this case, the recognition of off-line 
handwriting is more complex than the on-line case due 
to the presence of noise in the image acquisition 
process and the loss of temporal information such as the 
writing sequence and the velocity. 

This information is very helpfiil in the recognition 
process. Off-line and on-line recognition systems are 
also discriminated by the applications they are devoted 
to. 



The off-line recognition is dedicated to bank check 
processing, mail sorting, reading of routine commercial 
forms. The on-line recognition is mainly dedicated to 
pen computing industry and security domains such as 
signature verification and author authentication. This 
work is limited to the off-line recognition. 

For more than 35 years researchers have been 
working on an OCR system based on shape that has the 
accuracy of human vision. In order to facilitate future 
progress in document analysis, there is a need for a 
number of scanned document datasets, each 
representing a different class of documents, text, 
engineering drawings, addressed forms, handwritten 
manuscripts etc. 

2. STAGES 

The OCR system for hand written numeral used in 
this work has the following stages. 

2.1 Preprocessing 

The raw data is subjected to a number of 
preliminary processing steps to make it usable in the 
descriptive stages of character analysis. Pre-processing 
aims to produce data that are easy for the OCR systems 
to operate accurately. The main objectives of pre- 
processing are: 

• Noise Reduction 

• Stroke Width Normalization 
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Segmentation 
Thinning 



Global transformations and moments 



In the preprocessor stage smoothing and 
normalization is performed. Filling and thinning to 
eliminate noise, removal of isolated pixels and 
skeletonization is done in smoothing. 

Writing styles differ from person to person, hence 
there is a large variation in the character / numerals and 
also writing style of a person varies with times. Hence 
in a character / numeral recognition it requires a large 
database to account for the different styles of writing. 
A dataset of about 40 samples of each numeral has been 
created. Each sample is scaled to fit into a 64x64 
window and then thinning algorithm is applied to obtain 
the thinned / skeletonized image, which is used in 
feature extraction. 

The thin-line representation of certain elongated 
patterns, for example characters, would be closer to the 
human conception of these patterns; therefore, they 
permit a simpler structural analysis and more intuitive 
design of recognition algorithms. In more practical 
terms, thin-line representations of elongated patterns 
would be more amenable to extraction of features such 
as end points, junction points and coimections among 
the components whereas, some algorithms used in 
pattern recognition tasks also require one pixel wide 
line as input. For thiiming algorithm to be really 
effective, it should ideally compress data, retain 
significant features of the pattern and eliminate local 
noise without infroducing distortions of its own. 
Thinning is normally applied to binary images, and 
produces another binary image as output. The deletion 
of a pixel 'p' would depend on the configuration of 
pixels in a local neighborhood containing 'p'. 





Figure 1: Sample of Thirmed Character 



2.2 Feature Extraction 

In feature extraction stage each character is 
represented as a feature vector, which becomes its 
identity. The major goal of feature extraction is to 
exfract a set of features, which maximizes the 
recognition rate with the least amount of elements. 
Feature extraction methods are based on 3 types of 
features: 

• Statistical 

• Structural 
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Figure 2: Typical Features of Character. 

Characters can be represented by structural 
features with high tolerance to distortions and style 
variations. This type of representation may also encode 
some knowledge about the structure of the object or 
may provide some knowledge as to what sort of 
components make up that object. 

Structural features are based on topological and 
geometrical properties of the character, such as aspect 
ratio, cross points, loops, branch points, strokes and 
their directions, inflection between two points, 
horizontal curves at top or bottom, etc. 

Feature extraction forms the backbone of the 
recognition process. Features vary from global analysis. 
Structural analysis includes features such as junction 
points, end points, loops etc. These features are unique 
to each numeral. The junction points and end points 
form the characteristic points. For example, numeral 6 
has one jimction point and one end point and numeral 8 
may have single junction point or two junction points. It 
totally depends on how the numeral is written. 

Extraction of characteristic points can be of; 

• Transition function 

• End points 

• Junction points 

• Conic 's representation 

• Directional codes 

Transition Function: Each pixel Pi of a skeleton, 
resulting from the thinning process, has a transition 
function T (Pi) associated to it. T(Pi) represents the 
coimectivity between Pi and its eight neighbours. T(Pi) 
is defined as the number of transitions of (white) to 
(black) when the eight neighbours of Pi ( i.e. PI, P2, 
. . ., P8) are traced in a clockwise direction. 

End Points: 

An end point is a pixel Pi with T(Pi) = 1 . 

Junction Points: A junction point is a pixel Pi with T 
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(Pi) = 3. After the thinning process, the numeral tracing 
is ready and we find the junction points and end points. 

Using the characteristic points the pats are 
evaluated using Conic 's representation and directional 
code. 



3. CLASSIFIER 

Once the character has been represented as a 
feature vector, a classification rule has to be defined in 
order to classify it into one of the letters. Numerous 
techniques for off-line handwriting recognition have 
been investigated based on four general approaches of 
pattern recognition, such as template matching, 
statistical techniques, structural techniques and neural 
networks. 

Statistical techniques are concerned with 
statistical decision functions and a set of optimal 
criteria, which determine the probability of the 
observed pattern belonging to a certain class. Several 
popular handwriting recognition approaches belonging 
to this domain are: 

• k-Nearest-Neighbour (k-NN) 

• Bayesian 

. Hidden Markov Model (HMM) 

• Fuzzy set reasoning 

• Support Vector Machine (SVM) 

An HMM is called discrete if the observations are 
naturally discrete or quantized vectors from a codebook 
or continuous if these observations are continuous. 
HMM is been proven to be one of the most powerful 
tools for modeling speech and a wide variety of other 
real-world signals. These probabilistic models offer 
many desirable properties for modeling characters or 
words. One of the most important properties is the 
existence of efficient algorithms to automatically train 
the models without any need of labeling pre-segmented 
data. 

In order to define an HMM completely, the 
following elements are needed. 

• The number of states of the model, N. 

• The number of observation symbols in the 
alphabet, M. If the observations are continuous 
then M is infinite. 

• A set of state transition probabilities 



A = {aij} 
aij=P {qt = i}, 1 ^N, 

Where, q, denotes the current state. 

Transition probabilities should satisfy the normal 

stochastic constraints, 

ay >0, 1 < i,j <N and 

£35= 1, 1 < 1 < N 
j=i 

A probability distribution in each states, 

B = {bj(k)} 
Bj(k) =p{ot= vMt =j}. 1 <J <N. 1 <k<M 

Where, vk denotes the ^h observation symbol in the 
alphabet and the current parameter vector. 

The initial state distribution, 

71 = { Tli } where, 
TTi = p{qi = i}, 1 < i <N 

Therefore we can use the compact notation 
A. = (a, B, n) 

to denote an HMM with discrete probability 
distributions, while 

A i'^) ^Jm> Mjm> ^Jmr ^ 

denotes one with continuous densities. 

In this work each numeral is modeled by one 
HMM. The numeral "1" written is modeled by a small 
number of states. 
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Figure 3. Typical Features of Character 
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4. EXPERIMENTAL RESULTS 

Standardized downloadable handwritten datasets 
compiled by National Institute of Standards and 
Technology (NIST), are used in the experimental work, 
which is being used by several researchers. Special 
database 19 contains NlST's entire corpus of training 
materials for hand printed document and character 
recognition. It has published hand printed sample 
forms from 3600 writers, 810,000 character images 
isolated from their forms. As described earher, we 
have used a size normahzation step followed by a 
thinning step before feature extraction. 

The model has been tested for about 40 samples 
of each numeral to 9 and the number of correct 
recognition is physically recorded. The accuracy of 
recognition of various numerals is plotted in figure 4. 



Variation of accuracy of different numerals plotted, 
which indicates the Number 4 being the least accurate 
recognizable character and the number being the 
highest is shown is figure 5. 
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Figure 5 Variation of Accuracy 

5. CONCLUSION 

The objective of handwriting recognition is to 
have machines, which can read any text with the same 
recognition accuracy as humans, but at a faster rate. In 
the present work, a novel method is devised and used 
for thinning and feature extraction. In the feature 
extraction, structural features such as junction points, 
end points and loops have been predominantly used for 
numerals on a large data set. The features have been 
used to get HMM model. 

The model has been tested considering various 
numerals from the database and the recognition rate is 
between 84.5%. It is felt that, finding additional hidden 
unique features in the numeral can further increase the 
recognition rate. The thinning algorithm can be further 
improved by incorporation of heuristic rules to prune 
the spurious branches in the skeletal image. This will 
augment to improve the recognition rate of the HMM 
developed. A combination of HMM along with neural 
network classifier can also be considered. 

REFERENCES 

[1] Amin.A. (1997). "Offline Arabic Character 
Recognition: A survey". International Conference on 
Document Analysis and Recognition, vol. 2, pp. 596- 
599. 

[2] A. S. Britto, R. Sabourin, F. Bortolozzi, C. Y. Suen, 
"Foreground and Background Information in an 
HMM-Based Method for Recognition of Isolated 
Characters and Numeral Strings", 9th International 
Workshop on Frontiers in Handwriting Recognition 
(IWFHR-9), 2004, pp. 371-376. 

[3] Augustin, E., Baret, 0., Price, D. and Knerr, S. (1998). 
"Legal Amount Recognition on French Bank Checks 
Using a Neural Network-Hidden Markov Model 
Hybrid". International Workshop on Frontiers in 
Handwriting Recognition, pp. 45-54. 

[4] Bahl, L.R., Brown, P.F., Souze, P.V.de, and Mercer, 
R.L. (1992). "Maximum Mutual Information 
Estimation of Hidden Markov Model Parameters for 
Speech Recognition". International Conference on 
Acoustic, Speech and Signal Processing, pp. 49-52. 



[5] Bengio, Y. Mori, R.De Flammia, G. Kompe R. 
(1992). "Global Optimization of a Neural Network- 
Hidden Markov Model Hybrid", IEEE Transactions 
on Neural Networks, vol. 3, pp. 252-258. 

[6] Bishop, CM. (1995). Neural Networks for Pattern 
Recognition. Oxford University Press. 

[7] Blumenstein .M, and Verma .B. (2001). "Analysis of 
Segmentation on the CEDAR Benchmark Database". 
International Conference on Document Analysis and 
Recognition, pp. 1 142-1 146. 

[8] Bodenhausen .U , Manke. S and Waibel .A(1993). 
"Connectionist Architectural Learning for High 
Performance Character and Speech Recognition". 
International Conference on Acoustic, Speech and 
Signal Processing, vol. 1, pp. 625-628. 

[9] Bourlard .H., and Wellekens. C.I. (1990). "Links 
between Markov Models and Multilayer 
Perceptrons". IEEE Transactions on Pattern Analysis 
and Machine Intelligence, vol. 12, no 12, pp. 1167- 
1179. 

[10] Breuel .T.M. (1994b). "Design and Implementation of 
a System for the Recognition of Handwritten 
Responses on US Census Forms". International 
Workshop on Document Analysis Systems, Oct 18- 
20. 

[11] Bridle .I.S. (1990). "Probabilistic Interpretation of 
Feedforward Classification Network Outputs, with 
Relationships to Statistical Pattern Recognition". 
Neurocomputing - Algorithms, Architectures and 
Applications, vol. 6, pp. 227-236. 

[12] M. Brown, T. H. Fay, and C. L. Walker, 
"Handprinted symbol recognition system," Pattern 
Recognition, vol. 21, no. 

[13] Britto .Ir, R. Sabourin, F. Bortolozzi, and C. Y. Suen 
(2001). A two-stage HMM based system for 
recognizing handwritten numeral strings. In Proc. 6th 
International Conference on Document Analysis and 
Recognition, Seattle-USA, September, pp. 396-400. 

[14] Bunke .H, Roth. M., and Schukat-Talamazzini, E.G. 
(1995). "Off-line cursive Handwriting Recognition 
using Hidden Markov Models". Pattern Recognition, 
28(9): 1399-1413. 

[15] Caesar, T., Gloger, I.M. and Mandler, E. (1993a). 
"Preprocessing and Feature Extraction for a 
Handwriting Recognition System". International 
Conference on Document Analysis and Recognition, 
pp. 408-411. 

[16] Caesar, T., Gloger, I.M., and Mandler, E. (1993b). 

"Design of a System for Off-Line Recognition of 

Handwritten Word Images". lET POSTE. 
[17] Caesar, T., Gloger, I.M., and Mandler, E. (1995). 

"Estimating the Baseline for Handwritten Material". 

International Conference on Document Analysis and 

Recognition, pp. 382-385. 
[18] Cho W., Lee S.W., and Kim, I.H. "Modeling and 

Recognition of Cursive Words with Hidden Markov 

Models", Pattern Recognition 1995; 28:1941-1953 
[19] C-L Liu, K. Narukawa (2004b). Normalization 

Ensemble for Handwritten Character Recognition. 9th 

International Workshop on Frontiers of Handwriting 

Recognition, pp. 69-74 



12 An Efficient Hidden Markov Model... 



[20] Dcnkcr, J.S., and Burgcs, C.G.J. (1995). "Image 
Segmentation and Recognition". The Mathematics of 
Generalization, D. H. Wopert (ed.), Addison- Wesley, 
pp. 409-436. 

[21] Dietterich, T. G. (2000). "Ensemble methods in 
machine learning". First International Workshop on 
Multiple Classifier Systems, pp. 1-15. 

[22] Ding, Y., Kimura, F., Miyakc, Y., and Sridhar, M. 
(2000). "Slant Estimation for Handwritten Words by 
Directionally Refined Chain Code". International 
Workshop on Frontiers in Handwriting Recognition, 
pp. 53-62 

[23] Duda, R.O., Hart, P.E., and Stork, D.G. (2001). 
"Pattern Classification". 2nd Edition, John Wiley & 
son. 

[24] El Yacoubi, M. Gilloux, R. Sabourin, C. Y. Suen 
(1999). An hmm-based approach for offline 
unconstrained handwritten word modeling and 
recognition. IEEE Transactions on Pattern Analysis 
and Machine Intelligence, 21(8):752-760. 

[25] Gilloux, M., Lcmaric, B., and Leroux, M. (1995b). "A 
Hybrid Radial Basis Function/Hidden Markov Model 
Handwritten Word Recognition System". 
International Conference on Document Analysis and 
Recognition, Montreal, pp. 394-397. 

[26] Gorski, N. (1997). "Optimizing Error-Reject Trade 
off in Recognition Systems". Int. Conf on Document 
Analysis and Recognition, Ulm, pp. 1092-1096. 

[27] Guillevic, D.and Suen, C.Y. (1995). "Cursive Script 
Recognition Applied to the Processing of Bank 
Cheques". Proceedings of International Conference 
on Document Analysis and Recognition, Montreal, 
pp. 11-14. 

[28] Ho- Yon Kim, Kil-Taek Lim and Yun Seok Nam, 
"Handwritten Numeral Recognition Using Neural 
Network Classifier Trained with Negative Data", 
Proc. 8th International Workshop on Frontiers in 
Handwriting Recognition (IWFHR), Ontario, Canada, 
6-8 Aug. 2002, 395-400. 

[29] J.J. Hull, "A Database for Handwritten Text 
Recognition Research," IEEE Trans. Pattern Analysis 
and Machine Intelhgence, vol. 16, pp. 550-554, 1994. 

[30] K. Jain, R. P. W. Duin, J. Mao (2000). Statistical 
pattern recognition: A review. IEEE Transactions on 
Pattern Analysis and Machine Intelligence, 22(1):4- 
37. 

[31] Kimura, M. Shridhar (1992). Segmentation- 
recognition algorithm for handwritten numeral 
strings. Machine Vision Applications, No. 5, pp. 199- 
210. 

[32] Kundu, Y. He, M. Chen (2002). Alternatives to 
variable duration hmm in handwriting recognition. 
IEEE Transactions on Pattern Analysis and Machine 
Intelhgence, 20(11):1275-1280. 

[33] L. S. Oliveira, R. Sabourin (2004). Support Vector 
Machines for Handwritten Numerical String 
Recognition, 9th International Workshop on Frontiers 
in Handwriting Recognition, October 26-29, 
Kokubunji, Tokyo, Japan, pp 39-44. 

[34] L. Vuurpijl, L. Schomaker, M. van Erp (2003). 
Architectures for detecting and solving conflicts: two- 
stage classification and support vector classifiers. Int. 



Journal on Document Analysis and Recognition, 

5(4):213-223. 

[35] M. Hanmandlu and O.V. Ramana Murthy, "Fuzzy 
model based recognition of handwritten numerals". 
Pattern Recognition, vol.40, pp. 1840-1854, 2007. 

[36] M. Morita, L. S. Oliveira, R. Sabourin (2004). 
Unsupervised Feature Selection for Ensemble of 
Classifiers. IWFHR-9, pages 81-86. 

[37] Rabiner, L.R., and Juang, B.H. (1986). "An 
Introduction to Hidden Markov Models". IEEE ASSP 
Magazine, pp. 4-16. 

[38] Richard M., and Lippmann, R. (1991). "Neural 
Network Classifiers Estimate Bayesian a Posteriori 
Probabilities". Neural Computation, 3:461-483. 

[39] R. Plamondon and S. N. Srihari, "On-Line and off- 
line handwritten recognition: A comprehensive 
survey", IEEE Trans on RAMI, vol.22, pp.62-84, 
2000. 

BIOGRAPHY 

B.S.Saritha graduated fi-om SJ College of Engineering from 
Mysore. She is fourth semester M.Tech student of CMR 
Institute of Technology, Bangalore. Her field of interest is 
image processing and Data Mining. 

Hemanth S is working as Assistant Professor in RNS 
Institute of Technology, Bangalore since 2004. He 
completed his post graduation fi-om SJCE, Mysore. His area 
of interest are Image Processing , Data Mining and business 
intelligence. 



